Proposal for an Algorithm to Improve a Rational Policy in POMDPs

Reinforcement learning is a kind of machine learning. Partially Observable Markov Decision Process (POMDP) is a representative class of non-Markovian environments in reinforcemnet learning. We know the Rational Policy Making algorithm (RPM) to learn a deterministic rational policy in POMDPs. Though RPM can learn a policy very quickly, it needs numerous trials to improve the policy. Furthermore RPM does not apply the class where there is no deterministic rational policy. In this paper, we propose the Rational Policy Improvement algorithm (RPI) that combines RPM and the mark transit algorithm with χgoodness-of-fit test. RPI can learn a deterministic or stochastic rational policy in POMDPs. RPI is applied to maze environments. We show that RPI can learn the most stable rational policy in comparison with other methods.